Machine learning model generation device, machine learning model generation method, program, inspection device, inspection method, and printing device

ABSTRACT

There is provided a machine learning model generation device, a machine learning model generation method, a program, an inspection device, an inspection method, and a printing device for inspecting a defect of a printed matter with high accuracy. A machine learning model for detecting the defect of the printed matter is generated by using, as learning input information, at least learning inspection data based on a captured image obtained by capturing an image of a printed matter as an inspection target and second learning reference data based on print digital data and using, as learning output information, at least learning defect information of the learning inspection data estimated by performing comparison processing of the learning inspection data and first learning reference data based on a captured image obtained by capturing the image of the printed matter.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of PCT InternationalApplication No. PCT/JP2019/026685 fled on Jul. 4, 2019 claiming priorityunder 35 U.S.C § 119(a) to Japanese Patent Application No. 2018-139555filed on Jul. 25, 2018. Each of the above applications is herebyexpressly incorporated by reference, in its entirety, into the presentapplication.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a machine learning model generationdevice, a machine learning model generation method, a program, aninspection device, an inspection method, and a printing device, andparticularly relates to a technique of detecting a defect from a printedmatter.

2. Description of the Related Art

In printing of a printed matter, a defect such as ink loss, inkdripping, a scratch, and a streak may occur. For this reason, automaticinspection for detecting a defect of a printed matter is performed.

JP2014-186442A discloses an inspection device that determines an imagequality of a print image by calculation using, as parameters, featuredata of print image data, which is generated by reading a print imageprinted based on input data, and feature data of a reference image basedon the input data.

SUMMARY OF THE INVENTION

The device described in JP2014-186442A has a problem that imagestructures of the print image and the input data are different due to adifference of image generation sources of the print image and the inputdata and thus it is difficult to define feature data with highdetermination accuracy.

The present disclosure has been made in consideration of the abovecircumstances, and an object of the present disclosure is to provide amachine learning model generation device, a machine learning modelgeneration method, a program, an inspection device, an inspectionmethod, and a printing device capable of inspecting a defect of aprinted matter with high accuracy.

In order to achieve the above object, according to an aspect of thepresent disclosure, there is provided a machine learning modelgeneration method for detecting a defect of a printed matter bycomparing, using a machine learning model, inspection data which isacquired based on a captured image obtained by capturing an image of theprinted matter and reference data which is acquired based on printdigital data, the method including: an acquisition step of acquiringlearning inspection data that is based on a captured image obtained bycapturing an image of a printed matter as an inspection target which isprinted based on learning print digital data, learning defectinformation of the learning inspection data that is estimated byperforming comparison processing of first learning reference data andthe learning inspection data, the first learning reference data beingbased on a captured image obtained by capturing an image of a printedmatter as a reference which is printed based on the learning printdigital data, and second learning reference data based on the learningprint digital data; and a generation step of generating the machinelearning model by using at least the learning inspection data and thesecond learning reference data as learning input information and usingat least the learning defect information as learning output information.

According to the aspect, it is possible to generate a machine learningmodel for inspection using print digital data, and thus it is possibleto inspect a defect of a printed matter with high accuracy. The machinelearning model generation method corresponds to a machine learning modelmanufacturing method.

Preferably, the learning defect information includes a discrete value,and, in the generation step, the machine learning model for performingclassification is generated. Thereby, it is possible to inspect a defectof a printed matter with high accuracy.

Preferably, the discrete value is a binary discrete value indicating thepresence or absence of the defect of the printed matter. Thereby, it ispossible to inspect a defect of a printed matter with high accuracy.

Preferably, the discrete value is a ternary or higher discrete valueindicating a degree of the defect of the printed matter. Thereby, it ispossible to inspect a defect of a printed matter with high accuracy.

Preferably, the learning defect information includes a continuous value,and, in the generation step, the machine learning model for performingregression is generated. Thereby, it is possible to inspect a defect ofa printed matter with high accuracy.

Preferably, the learning defect information includes positioninformation of the defect of the printed matter. Thereby, it is possibleto inspect a defect of a printed matter with high accuracy.

Preferably, in the generation step, the machine learning model isgenerated by deep learning. Thereby, it is possible to properly generatea machine learning model.

Preferably, in the generation step, the machine learning model isgenerated by using at least the first learning reference data and thesecond learning reference data as learning input information and usingat least the learning defect information indicating that a defect doesnot exist as learning output information. Thereby, it is possible tolearn a case where a defect does not exist.

Preferably, in the acquisition step, at least the learning inspectiondata is acquired, the learning inspection data being obtained bycapturing an image of a printed matter printed based on processed printdigital data in which a defect is expressed by processing at least apart of the learning print digital data. Thereby, it is possible tocollect a large amount of learning data.

Preferably, in the acquisition step, the first learning reference datais acquired, and the method further includes a comparison processingstep of estimating the learning defect information of the learninginspection data by performing comparison processing of the learninginspection data and the first learning reference data. Thereby, it ispossible to properly acquire the learning defect information.

Preferably, in the comparison processing step, comparison processing isperformed by using a comparison-processing machine learning model.Thereby, it is possible to properly estimate the learning defectinformation.

Preferably, the machine learning model generation method furtherincludes: a sensory evaluation value input step of inputting a sensoryevaluation value obtained by comparing a first printed matter as areference which is printed based on the learning print digital data anda second printed matter as a comparison target which is printed based onthe learning print digital data; and a comparison-processing modelgeneration step of generating the comparison-processing machine learningmodel by using comparison-processing learning reference data obtained bycapturing an image of the first printed matter and comparison-processinglearning inspection data obtained by capturing an image of the secondprinted matter as learning input information and using the sensoryevaluation value as learning output information. Thereby, it is possibleto properly acquire the learning defect information.

In order to achieve the above object, according to another aspect of thepresent disclosure, there is provided an inspection method including: adefect inspection step of acquiring inspection data based on a capturedimage obtained by capturing an image of a printed matter as aninspection target which is printed based on print digital data andreference data based on the print digital data and detecting a defect ofthe printed matter as the inspection target by comparing the inspectiondata and the reference data by using the machine learning model.

According to the aspect, it is possible to inspect a defect of a printedmatter with high accuracy.

In order to achieve the above object, according to still another aspectof the present disclosure, there is provided a machine learning modelgeneration device for detecting a defect of a printed matter bycomparing, using a machine learning model, inspection data which isacquired based on a captured image obtained by capturing an image of theprinted matter and reference data which is acquired based on printdigital data, the device including: an acquisition unit that acquireslearning inspection data that is based on a captured image obtained bycapturing an image of a printed matter as an inspection target which isprinted based on learning print digital data, learning defectinformation of the learning inspection data that is estimated byperforming comparison processing of first learning reference data andthe learning inspection data, the first learning reference data beingbased on a captured image obtained by capturing an image of a printedmatter as a reference which is printed based on the learning printdigital data, and second learning reference data based on the learningprint digital data; and a generation unit that generates the machinelearning model by using at least the learning inspection data and thesecond learning reference data as learning input information and usingat least the learning defect information as learning output information.

According to the aspect, it is possible to generate a machine learningmodel for inspection using print digital data, and thus it is possibleto inspect a defect of a printed matter with high accuracy.

In order to achieve the above object, according to still another aspectof the present disclosure, there is provided an inspection deviceincluding: the machine learning model generation device described in theabove; and a defect inspection unit that acquires inspection data basedon a captured image obtained by capturing an image of a printed matteras an inspection target which is printed based on print digital data andreference data based on the print digital data and detects a defect ofthe printed matter as the inspection target by comparing the inspectiondata and the reference data by using a machine learning model.

According to the aspect, it is possible to generate a machine learningmodel for inspection using print digital data, and thus it is possibleto inspect a defect of a printed matter with high accuracy.

In order to achieve the above object, according to still another aspectof the present disclosure, there is provided a printing deviceincluding: the inspection device; a printing unit that generates aprinted matter by performing printing based on the print digital data; acamera that captures an image of the printed matter; and an output unitthat outputs a detection result of a defect of the printed matter.According to the aspect, it is possible to inspect a defect of a printedmatter, which is generated, with high accuracy.

Preferably, the printing device further includes a processing unit thatgenerates processed print digital data in which a defect is expressed byprocessing at least a part of the learning print digital data. Further,preferably, the printing unit generates a printed matter with a defectby performing printing based on the processed print digital data, thecamera captures an image of the printed matter with a defect, and theacquisition unit acquires, as the learning inspection data, at leastdata based on a captured image obtained by capturing an image of theprinted matter with a defect. Thereby, it is possible to collect a largeamount of learning inspection data.

Preferably, the generation unit generates an adjusted machine learningmodel suitable for the printing device by adjusting a machine learningmodel by using learning inspection data based on a captured imageobtained by capturing an image of a printed matter by the camera.Thereby, it is possible to perform inspection according to a printcondition of a customer of the printing device.

Preferably, the printing unit performs printing using an ink jet head.According to the aspect, it is possible to perform inspection of aprinted matter obtained by using an ink jet head.

In order to achieve the above object, according to still another aspectof the present disclosure, there is provided a program for causing acomputer to execute the machine learning model generation method.

According to the aspect, it is possible to generate a machine learningmodel for inspection in which comparison with print digital data isperformed, and thus it is possible to inspect a defect of a printedmatter with high accuracy.

According to the present disclosure, it is possible to inspect a defectof a printed matter with high accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a machinelearning model generation system.

FIG. 2 is a flowchart illustrating a machine learning model generationprocedure.

FIG. 3 is a block diagram illustrating a configuration of aprinted-matter inspection device.

FIG. 4 is a flowchart illustrating a machine learning model useprocedure.

FIG. 5 is a block diagram illustrating a configuration of a machinelearning model generation system.

FIG. 6 is a flowchart illustrating a machine learning model generationprocedure.

FIG. 7 is a block diagram illustrating a configuration of a machinelearning model generation system.

FIG. 8 is a flowchart illustrating a comparison-processing machinelearning model generation procedure.

FIG. 9 is a block diagram illustrating a configuration of a printingdevice including a learning information generation device.

FIG. 10 is a flowchart illustrating collection of learning defectinformation.

FIG. 11 is a block diagram illustrating a configuration of a printingdevice that generates and updates a machine learning model.

FIG. 12 is a flowchart illustrating generation and update of a machinelearning model.

FIG. 13 is a block diagram illustrating a configuration of a printingdevice that generates and updates a machine learning model.

FIG. 14 is a block diagram illustrating an internal configuration of anink jet printing device.

FIG. 15 is a diagram illustrating an example of a printed matter whichis printed by an ink jet printing device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings.

<Correct Answer Data Comparison Method and Original Data ComparisonMethod>

Inspection of a printed matter is performed by obtaining inspection databy reading the printed matter as an inspection target by an imagecapturing device, and comparing the inspection data with reference dataas a reference of inspection. In this specification, this comparisonprocessing is classified into a correct answer data comparison methodand an original data comparison method.

In the correct answer data comparison method, comparison with referencedata generated from a printed matter is performed. The reference data isacquired based on a captured image obtained by capturing an image of aprinted matter having no defect by an image capturing device. By settingan image capturing condition of the reference data to be the same imagecapturing condition as an image capturing condition of the inspectiondata, it is possible to relatively reduce an inspection difficulty levelas compared with a case of comparing pieces of data generated fromdifferent sources.

On the other hand, because it is necessary to capture an image of aprinted matter having no defect by an image capturing device, there is aproblem that a correct answer data comparison method cannot be used in ajob for only one page and variable print in which the number of printpages of each machine is one page. In addition, it is necessary todefine how to obtain the reference data for each print type such asnormal print, gathering print, and back-surface print, and as a result,there is a problem that the conditions become complicated.

Further, in a case where there is a defect in the reference data, insome cases, non-detection in which a portion of the inspection data witha defect is not determined as a defect may occur, or erroneous detectionin which a portion of the inspection data without a defect is determinedas a defect may occur.

In order to deal with these situations, it is necessary to adopt theoriginal data comparison method. In the original data comparison method,comparison with the acquired reference data is performed withoutprinting. The reference data is acquired from digital image data whichis input to a printing device, or is acquired by performing, on thedigital image data, various pre-processing such as resolutionconversion, color conversion, gradation conversion, screeningprocessing, and statistical processing.

In the original data comparison method, it is possible to perform aninspection in a job for only one page and variable print, which cannotbe realized in the correct answer data comparison method. Further, it iseasy to correspond to each print type.

On the other hand, since the inspection data and the reference data aregenerated from different data sources, characteristics of the inspectiondata and the reference data are largely different. As a result, there isa problem that an inspection difficulty level of the original datacomparison method is relatively high as compared with the correct answerdata comparison method.

<Use of Machine Learning Technique and Ensuring of Learning Data>

In recent years, machine learning techniques have made great progress,and under a condition that there is a large amount of high qualitylearning data, it is known that pattern recognition by latest machinelearning such as deep learning can obtain better performance thanpattern recognition using features designed by humans.

In order to obtain high performance of defect inspection in deeplearning, a large amount of high quality learning data is required. Onthe other hand, in a case of acquiring a degree of a defect by humansensory evaluation, a large number of man-hours is required for sensoryevaluation, and as a result, there is a problem that it is difficult toacquire a large amount of learning data. Further, in sensory evaluation,evaluation values are likely to vary, and as a result, there is aproblem that it is difficult to acquire high-quality learning data.

Therefore, by performing defect inspection by the correct answer datacomparison method, a large amount of high-quality learning data isrelatively easily acquired. The correct answer data comparison methodhas an advantage that the number of man-hours required for automaticevaluation is small and thus a large amount of learning data can beeasily acquired. Further, the correct answer data comparison method hasan advantage that a variation in result is small by objective evaluationand thus high-quality learning data can be easily acquired.

First Embodiment

A machine learning model generation system according to a firstembodiment generates a machine learning model for detecting a defect ofa printed matter by comparing inspection data, which is acquired basedon a captured image obtained by capturing an image of a printed matter,and reference data, which is acquired based on print digital data byusing a machine learning model.

In this specification, a term “defect” is a concept that includeseverything printed in a state where there is an unintended change froman original print state, such as ink loss, ink dripping, oil dripping,dust adhesion, other spot-shaped defects, unevenness, color variation,character loss, scratches, change in glossiness, wrinkles, streak-shapeddefects due to a defective nozzle of ink jet nozzles, streak-shapeddefects due to failure of correction of a defective nozzle, and paperdeformation such as wrinkles.

[Configuration of Machine Learning Model Generation System and MachineLearning Model Generation Procedure]

FIG. 1 is a block diagram illustrating a configuration of a machinelearning model generation system. FIG. 2 is a flowchart illustrating amachine learning model generation procedure.

The generation system 10 includes a learning information generationdevice 20 and a model generation device 40.

The learning information generation device 20 includes a first learningreference data storage unit 22, a learning inspection data storage unit24, a comparison processing unit 26, a learning defect informationstorage unit 28, and a second learning reference data storage unit 30.

The first learning reference data storage unit 22 is a memory thatstores first learning reference data D1. The first learning referencedata D1 is data based on a captured image obtained by capturing an imageof a printed matter without a defect (a printed matter having no defect)by an image capturing device (not illustrated).

The learning inspection data storage unit 24 is a memory that storeslearning inspection data D2. The learning inspection data D2 is databased on a captured image obtained by capturing an image of a printedmatter with a defect by an image capturing device (not illustrated) anddata based on a captured image obtained by capturing an image of aprinted matter without a defect by an image capturing device (notillustrated).

The first learning reference data D1 and the learning inspection data D2may be the captured images itself or may be data obtained by performingcertain pre-processing on the captured images. As pre-processing for thecaptured image, various image processing such as color conversion,gradation conversion, resolution conversion, and filtering may be used,or profile data generated by calculating a statistic value such as anaverage value, a median value, a maximum value, or a minimum value ofthe image in a vertical direction and a horizontal direction may beused. Further, feature data extracted by frequency analysis orstatistical analysis may be used.

The pre-processing may be performed on the captured image in thelearning information generation device 20, and the pre-processed firstlearning reference data D1 and the pre-processed learning inspectiondata D2 may be acquired from an external device via an interface (notillustrated).

In the first learning reference data D1 and the learning inspection dataD2, a pair of the first learning reference data D1 and the learninginspection data D2 respectively based on the same learning print digitaldata are present. That is, among printed matters printed based on thesame learning print digital data, the first learning reference data D1,which is acquired from a captured image of a printed matter without adefect, and the learning inspection data D2, which is acquired from acaptured image of a printed matter with a defect or a printed matterwithout a defect, form a pair. Here, the learning print digital datarefers to print digital data used for learning. Thus, the learning printdigital data is not limited to print digital data specially generatedfor learning, and may be print digital data used for inspection of theprinted matter.

The image capturing device used to acquire the first learning referencedata D1 and the learning inspection data D2 may be any image capturingdevice. For example, a line scanning camera may be used, or a camera inwhich optical elements are two-dimensionally arranged may be used. Amonochrome camera which cannot acquire color information may be used, ora camera which can acquire color information such as red, green, blue(RGB) information or spectral information may be used. An optimum cameramay be selected according to a defect to be detected.

Preferably, a plurality of pieces of the learning inspection data D2 anda plurality of pieces of the first learning reference data D1 pairedwith the learning inspection data D2 are acquired from the same imagecapturing device (device having the same serial number). Thereby, adifficulty level of comparison processing P1 to be described is reduced,and thus it becomes easy to obtain high-quality learning defectinformation D3 to be described.

It is not always necessary to acquire all of the plurality of pieces ofthe learning inspection data D2 and the plurality of pieces of the firstlearning reference data D1 paired with the learning inspection data D2from the same image capturing device. The pair of the first learningreference data D1 and the learning inspection data D2 may be acquiredfrom different image capturing devices having almost the same imagecapturing characteristics. For example, in a case where a plurality ofimage capturing devices, which have substantially the same opticalelement characteristics, substantially the same lightingcharacteristics, and substantially the same geometrical conditions ofelements and have substantially the same image capturingcharacteristics, are prepared and work is performed in parallel, theplurality of pieces of the learning inspection data D2 and the pluralityof pieces of the first learning reference data D1 paired with thelearning inspection data D2 can be acquired in a short time and in largequantity.

Further, in order to acquire a large number of images, image capturingdevices having different image capturing characteristics may be used. Amachine learning model generated in this case has characteristics thatare robust to image capturing characteristics, and is used for machinelearning. Thus, in a machine learning model use procedure, theinspection data can be acquired by using the image capturing deviceshaving various image capturing characteristics. Here, in a case whereinspection performance is prioritized, it is desirable to use imagecapturing devices having almost the same image capturingcharacteristics.

Similarly, the printing device used to acquire the first learningreference data D1 and the learning inspection data D2 may be anyprinting device. Further, preferably, a plurality of pieces of thelearning inspection data D2 and a plurality of pieces of the firstlearning reference data D1 paired with the learning inspection data D2are acquired from the same printing device. On the other hand, thelearning inspection data D2 and the first learning reference data D1 maybe acquired from different printing devices.

The comparison processing unit 26 is an arithmetic unit that performscomparison processing P1 as defect inspection of a printed matter. Thecomparison processing P1 is processing of estimating learning defectinformation D3 by comparing a pair of the first learning reference dataD1 and the learning inspection data D2.

Both of the first learning reference data D1 and the learning inspectiondata D2 are based on a captured image captured by the image capturingdevice. Thus, in the comparison processing P1, even in a case wheresimple comparison processing of the correct answer data comparisonmethod is used, defect inspection can be performed with relatively highaccuracy and with few non-detection and erroneous detection. Further,the comparison processing P1 is automatic inspection, and thus a largeamount of learning defect information D3 can be easily acquired bypreparing images.

For example, in a case where the first learning reference data D1 is thecaptured image itself and the learning inspection data D2 is thecaptured image itself, as a specific example of the comparisonprocessing P1, the following processing may be considered. That is, inthe comparison processing P1, a defect portion of the printed matter isextracted by obtaining a difference image between the first learningreference data D1 and the learning inspection data D2 and performingthreshold value processing on the obtained difference image. Further, inthe comparison processing P1, the learning defect information D3 isoutput from the extracted defect portion. The comparison processing P1is not limited to processing using difference processing and thresholdvalue processing, and processing of obtaining the learning defectinformation D3 may be appropriately applied.

For example, as the learning defect information D3, the followinginformation (1) to (7) may be applied.

(1) Binary Information Indicating Presence or Absence of Defect

A binary discrete value indicating the presence or absence of the defectof the printed matter may be used. For example, in a case where there isno defect (in a case where a defect does not exist), the binary discretevalue is defined as 0, and in a case where there is a defect, the binarydiscrete value is defined as 1. The binary discrete value is used asdata classified by machine learning.

(2) Information Indicating Type of Defect

A ternary discrete value indicating type information of a defect may beused. For example, in a case where there is no defect, the ternarydiscrete value is defined as 0, in a case where there is a streak-shapeddefect, the ternary discrete value is defined as 1, and in a case wherethere is a spot-shaped (also referred to as point-shaped, speck-shaped,or dot-shaped) defect, the ternary discrete value is defined as 2. Thedefect types may be further increased.

(3) Information Indicating Degree of Defect

A degree of a defect is defined as continuous values. In a case wherethere is no defect, the continuous value is set to 0, and in a casewhere there is a defect, as a degree of the defect is larger, that is,as it is easier to visually recognize the defect, the continuous valueis set to a larger number. For example, a degree of a defect is definedas continuous values of 0 to 10. In this case, in the machine learningmodel, regression is performed.

Alternatively, a categorized ternary or higher discrete value indicatinga degree of a defect of a printed matter may be used. For example, in astate where a degree of a defect is defined by categorized ternarydiscrete values of 0, 1, and 2, in a case where there is no defect, adegree of the defect is set to 0, in a case where there is a smalldefect, a degree of the defect is set to 1, and in a case where there isa large defect, a degree of the defect is set to 2. In this case, in themachine learning model, classification is performed. The categorizedvalues may be ternary or higher discrete values.

(4) Information Indicating Position of Defect

Information including position information of the defect of the printedmatter may be used. For example, the information is defined as dataobtained by converting a position coordinate into a value.Alternatively, the information is defined as image information in whicha non-defect position is set to 0 and a defect position is set to 1.

(5) Image Information Including Both of Information Indicating Degree ofDefect and Information Indicating Position of Defect

For example, image information in which a non-defect position is set to0 and a continuous value or a discrete value indicating a degree valueof a defect is set to a defect position is defined.

(6) Image Information Including Both of Information Indicating Type ofDefect and Information Indicating Position of Defect

For example, image information in which a non-defect position is set to0 and information indicating a type of a defect is set to a defectposition is defined.

(7) Multi-Channel Image Information Including all of InformationIndicating Degree of Defect, Information Indicating Type of Defect, andInformation Indicating Position of Defect

For example, an image including a first channel in which informationindicating a degree of a streak-shaped defect and information indicatinga position of the streak-shaped defect are set is defined, and an imageincluding a second channel in which information indicating a degree of aspot-shaped defect and information indicating a position of thespot-shaped defect are set is defined.

The learning defect information D3 is not limited to the aboveinformation (1) to (7) as long as the information is informationindicating a characteristic of a defect. The learning defect informationD3 may have a plurality of types. For example, the learning defectinformation D3 may include (1) binary information indicating thepresence or absence of a defect and (4) information indicating aposition of a defect, or may include (1) binary information indicatingthe presence or absence of a defect, (3) information indicating a degreeof a defect, and (4) information indicating a position of a defect,pieces of the information being described in the above examples.

The learning defect information storage unit 28 is a memory that storesthe learning defect information D3 acquired by the comparison processingunit 26.

The second learning reference data storage unit 30 is a memory thatstores second learning reference data D4. The second learning referencedata D4 is data based on learning print digital data which is used in acase of printing the printed matters of the first learning referencedata D1 and the learning inspection data D2.

The second learning reference data D4 is not derived from a capturedimage obtained by capturing an image of a printed matter, but is thelearning print digital data itself or data obtained by performingcertain pre-processing on the learning print digital data. Aspre-processing for the learning print digital data, various imageprocessing such as color conversion, gradation conversion, resolutionconversion, and filtering may be used, or profile data generated bycalculating a statistic value such as an average value, a median value,a maximum value, or a minimum value of the image in a vertical directionand a horizontal direction may be used. Further, feature data extractedby frequency analysis or statistical analysis may be used.

The pre-processing may be performed on the learning print digital datain the learning information generation device 20, and the pre-processedsecond learning reference data D4 may be acquired from an externaldevice via an interface (not illustrated).

Further, the model generation device 40 includes an informationacquisition unit 42, a model generation unit 44, and a model storageunit 46.

The information acquisition unit 42 is an input interface that acquiresthe learning inspection data D2, the learning defect information D3, andthe second learning reference data D4 from the learning informationgeneration device 20. The information acquisition unit 42 acquires thelearning inspection data D2 and the second learning reference data D4 aslearning input information, and acquires the learning defect informationD3 as learning output information. Here, the learning input informationcorresponds to an explanation variable, and the learning outputinformation corresponds to an objective variable.

The model generation unit 44 is an arithmetic unit that performs modelgeneration processing P2. The model generation processing P2 isprocessing of generating a machine learning model D5 using an originaldata comparison method, from the learning inspection data D2 and thesecond learning reference data D4 as learning input information and thelearning defect information D3 as learning output information. In themodel generation processing P2, as the learning input information, atleast information based on the learning inspection data D2 andinformation based on the second learning reference data D4 may be used,and, as the learning output information, at least information based onthe learning defect information D3 may be used.

For machine learning, any method such as multiple regression analysis,support vector machine, a partial least-square method, and deep learningmay be used. In deep learning, feature data used for pattern recognitionis also automatically extracted. By using a large amount of thehigh-quality learning defect information D3 acquired by the learninginformation generation device 20, it is possible to generate a machinelearning model having high inspection performance. Here, in the modelgeneration processing P2, a machine learning model D5 is generated bydeep learning.

The learning inspection data D2 is used in both of the comparisonprocessing P1 in the learning information generation device 20 and themodel generation processing P2 in the model generation device 40. On theother hand, the learning inspection data D2 used for the comparisonprocessing P1 and the learning inspection data D2 used for the modelgeneration processing P2 do not necessarily have to be subjected to thesame pre-processing. That is, the learning inspection data D2 used forthe comparison processing P1 and the learning inspection data D2 usedfor the model generation processing P2 may be data obtained byperforming different pre-processing on a captured image obtained bycapturing an image of a printed matter without a defect by an imagecapturing device. The greatest advantage of the present embodiment isthat a large amount of the learning defect information D3 can beaccurately generated by the comparison processing P1. Thus, there is noproblem even in a case where different pre-processing is respectivelyperformed.

Further, the learning defect information D3 generated in the comparisonprocessing P1 and the learning defect information D3 used in the modelgeneration processing P2 do not necessarily have to be the same, and thelearning output information used in the model generation processing P2may be generated based on the learning defect information D3. Since thecomparison processing P1 and the model generation processing P2 aredifferent processing, the learning defect information D3 generated inthe comparison processing P1 and the learning defect information D3 usedin the model generation processing P2 may be defined in a formatsuitable for each processing.

Further, in the model generation processing P2, at least the learningoutput information generated based on the learning defect information D3may be used. For example, the learning output information generatedbased on sensory evaluation may be used together.

As described above, the learning defect information D3 may have aplurality of types such as the presence or absence of a defect and aposition of a defect. On the other hand, in this case, a plurality oftypes of the learning output information are also defined. A use methodin a case where the learning output information has a plurality of typeswill be described by taking deep learning as an example.

In deep learning, an input layer based on the learning input informationand an output layer based on the learning output information can bedefined. In deep learning, an intermediate layer such as a convolutionlayer and a pooling layer is defined between the input layer and theoutput layer, and various parameters in the intermediate layer areupdated by a technique called back propagation from an error in theoutput layer. Thus, learning is performed such that the error in theoutput layer is reduced. In addition, in deep learning, the layers canbe separated or combined in the middle. Thus, a plurality of outputlayers can be defined by separating the layers in the middle. Forexample, in a case where two output layers are defined, an error on thepresence or absence of a defect can be calculated in a first outputlayer, and an error on a position of a defect can be calculated in asecond output layer. Further, in a state where a sum of the two errorsis defined as the final error amount, by performing back propagation,the parameters are updated such that both of the error on the presenceor absence of a defect and the error on a position of a defect arereduced. In this way, the model generation processing P2 using theplurality of types of the learning output information can be performed.

The model storage unit 46 is a memory that stores the machine learningmodel D5 generated by the model generation unit 44.

The generation system 10 configured as described above generates themachine learning model D5 as follows (an example of a generation methodof the machine learning model). That is, the information acquisitionunit 42 acquires the first learning reference data D1, the learninginspection data D2, and the second learning reference data D4 (anexample of an acquisition step). In addition, the comparison processingP1 is performed on a pair of the first learning reference data D1 andthe learning inspection data D2, and thus the learning defectinformation D3 is acquired (an example of a comparison processing step).Further, the model generation unit 44 performs the model generationprocessing P2 of generating the machine learning model D5 by using, asthe learning input information, at least information based on thelearning inspection data D2 and information based on the second learningreference data D4 and using, as the learning output information, atleast information based on the learning defect information D3 (anexample of a generation step). Thereby, it is possible to generate amachine learning model capable of performing defect inspection using anoriginal data comparison method with high accuracy and with less effort.

It is desirable to acquire the first learning reference data D1 based ona captured image that does not include a defect as a detection target inall captured images. In a case where the first learning reference dataD1 includes a defect as a detection target, in the comparison processingusing the correct answer data comparison method, non-detection anderroneous detection may occur at a defect portion. As a result,erroneous information may be included in the learning defect informationD3, and this adversely affects performance of the machine learning modelD5 generated in the model generation processing P2.

In the learning defect information D3, in a case where a ratio of theerroneous information is sufficiently smaller than a ratio of normalinformation, the adverse effect may be small, and this may not be apractical problem. On the other hand, basically, erroneous informationgenerated in such a procedure should be excluded as much as possible. Asa method of confirming whether or not there is a defect in the firstlearning reference data D1, a method of visually confirming a printedmatter or a method of displaying a captured image on a display andvisually confirming the captured image may be used. The visualconfirmation may be performed only on the printed matter of the firstlearning reference data D1. The visual confirmation does not need to beperformed on the printed matter of the learning inspection data D2, andthus a burden of the visual check can be significantly reduced.

[Configuration of Printed-Matter Inspection Device and Machine LearningModel Use Procedure]

FIG. 3 is a block diagram illustrating a configuration of aprinted-matter inspection device. FIG. 4 is a flowchart illustrating amachine learning model use procedure.

The printed-matter inspection device 50 is a device that performsinspection of a printed matter by using the machine learning model D5.The printed-matter inspection device 50 includes an inspection datastorage unit 52, a reference data storage unit 54, a model storage unit56, a defect inspection unit 58, and an inspection result storage unit60.

The inspection data storage unit 52 is a memory that stores inspectiondata D6. The inspection data D6 is data based on a captured imageobtained by capturing an image of a printed matter as an inspectiontarget by an image capturing device.

Similar to the first learning reference data D1 and the learninginspection data D2, the inspection data D6 may be the captured imageitself or may be data obtained by performing certain pre-processing onthe captured image. The inspection data D6 may be generated from thecaptured image in the printed-matter inspection device 50, or may beacquired from an external device via an interface (not illustrated).

The inspection data D6 is acquired from an image capturing device. Onthe other hand, it is not always necessary to use the same imagecapturing device as the image capturing device used to acquire the firstlearning reference data D1 and the learning inspection data D2. In acase where an image capturing device, which has substantially the sameimage capturing characteristics as the image capturing device used inthe generation procedure of the machine learning model D5, is used,defect inspection having sufficient performance can be performed.

Further, in a state where, in the generation procedure of the machinelearning model D5, a model which is robust to image capturingcharacteristics can be generated from a large amount of learning data byusing image capturing devices having various image capturingcharacteristics, even in a case where image capturing devices havingdifferent image capturing characteristics are used in the model useprocedure, defect inspection having sufficient performance can beperformed.

The reference data storage unit 54 is a memory that stores referencedata D7. The reference data D7 is data based on print digital data usedin a case of printing a printed matter of the inspection data D6. Thereference data D7 may be print digital data itself, or may be dataobtained by performing certain pre-processing on the print digital data.As pre-processing for the print digital data, various image processingsuch as color conversion, gradation conversion, resolution conversion,and filtering may be used, or profile data generated by calculating astatistic value such as an average value, a median value, a maximumvalue, or a minimum value of the image in a vertical direction and ahorizontal direction may be used. Further, feature data extracted byfrequency analysis or statistical analysis may be used.

The pre-processing may be performed on the print digital data in theprinted-matter inspection device 50, and the pre-processed referencedata D7 may be acquired from an external device via an interface (notillustrated).

The model storage unit 56 is a memory that stores the machine learningmodel D5. The printed-matter inspection device 50 acquires the machinelearning model D5 stored in the model storage unit 46 of the modelgeneration device 40, and stores the machine learning model D5 in themodel storage unit 56.

The defect inspection unit 58 is an arithmetic unit that performs defectinspection processing P3. The defect inspection processing P3 isprocessing of performing defect inspection of a printed matter byapplying the inspection data D6 and the reference data D7, which areinput, to the machine learning model D5 and acquiring an inspectionresult D8.

The inspection result storage unit 60 is a memory that stores theinspection result D8.

The printed-matter inspection device 50 configured as described aboveperforms defect inspection as follows (an example of an inspectionmethod). That is, the defect inspection unit 58 performs defectinspection processing P3 by applying the inspection data D6 and thereference data D7, which are input, to the machine learning model D5,and generates an inspection result D8 (an example of a defect inspectionstep).

The defect inspection of the printed matter may be performed immediatelyafter the inspection data D6 is acquired by using an online imagecapturing device at the time of printing the printed matter, or may beperformed after the inspection data D6 is printed and is captured afterprinting. Alternatively, the defect inspection may be performed afterthe inspection data D6 is acquired by an offline image capturing deviceafter printing.

As described above, by providing the machine learning model D5 generatedby the model generation device 40 to the printed-matter inspectiondevice 50, defect inspection by the original data comparison method canbe performed, and defect inspection for a one-page printed matter suchas variable print can be performed.

In printing, in many cases, “paper” is used as a print medium. On theother hand, the print medium according to the present embodiment is notlimited to paper. For example, the present embodiment may be applied toa printed matter printed on a transparent medium sheet, or may beapplied to a printed matter printed on any other medium.

Second Embodiment

In the first embodiment, as the learning inspection data D2 without adefect, data based on a captured image obtained by capturing an image ofa printed matter without a defect by an image capturing device (notillustrated) is used. On the other hand, as the learning inspection dataD2 without a defect, the first learning reference data D1 may be used.This is because the first learning reference data D1 is generated basedon a captured image obtained by capturing an image of a printed matterwithout a defect.

[Configuration of Machine Learning Model Generation System and MachineLearning Model Generation Procedure]

FIG. 5 is a block diagram illustrating a configuration of a machinelearning model generation system. FIG. 6 is a flowchart illustrating amachine learning model generation procedure. The same portions as thosein FIGS. 1 and 2 are denoted by the same reference numerals, and adetailed description thereof will be omitted.

A generation system 62 uses, as the learning inspection data D2 withouta defect, the first learning reference data D1. In the generation system62, the first learning reference data storage unit 22 of the learninginformation generation device 20 and the information acquisition unit 42of the model generation device 40 are connected to each other. Theinformation acquisition unit 42 acquires, as the learning inputinformation, the first learning reference data D1.

The learning defect information D3 paired with the learning inspectiondata D2 without a defect may be generated by performing the comparisonprocessing P1, or may be generated without passing through thecomparison processing P1. In a case where the learning defectinformation D3 is generated without passing through the comparisonprocessing P1, learning defect information D3N without a defect isgenerated and input to the information acquisition unit 42. The reasonwhy the learning defect information D3N without a defect can begenerated without passing through the comparison processing P1 is that adefect does not exist.

For example, in a case where the learning defect information D3 is“information indicating a degree of a defect”, the comparison processingP1 is required to obtain a degree of a defect in a case where a defectexists. On the other hand, in a case where a defect does not exist, itis sufficient to generate information indicating that a degree of adefect is zero, and thus the comparison processing P1 is not alwaysrequired. Further, for example, in a case where the learning defectinformation is “information indicating a position of a defect”, thecomparison processing P1 is required to obtain a position of a defect ina case where a defect exists. On the other hand, in a case where adefect does not exist, it is sufficient to generate informationindicating that there is no defect at all positions, and thus thecomparison processing P1 is not always required.

Similar to the first embodiment, the model generation unit 44 performsmodel generation processing P2 based on the learning inspection data D2and the second learning reference data D4 as learning input informationand the learning defect information D3 as learning output information.In a case where the first learning reference data D1 is used as thelearning inspection data D2 without a defect and the comparisonprocessing P1 is not performed, the learning defect information D3Nwithout a defect is used.

By performing the model generation processing P2 in this way, it ispossible to generate a machine learning model D5 obtained by learning atype of data with a defect and a type of data without a defect.

In the present embodiment, the first learning reference data D1 is usedin both of the comparison processing P1 and the model generationprocessing P2. On the other hand, the first learning reference data D1used for the comparison processing P1 and the first learning referencedata D1 used for the model generation processing P2 do not necessarilyhave to be subjected to the same pre-processing. That is, the firstlearning reference data D1 used for the comparison processing P1 and thefirst learning reference data D1 used for the model generationprocessing P2 may be data obtained by performing differentpre-processing on a captured image obtained by capturing an image of aprinted matter without a defect by an image capturing device.

Third Embodiment

[Use of Sensory Value in Comparison Processing (Correct Answer DataComparison Method)]

A defect in printing cannot always be defined only by the presence orabsence of a defect as a physical quantity. For example, in asingle-pass ink jet printing device, a streak-shaped defect may occur ona printed matter due to curved ejection of each nozzle, and visibilityof the streak changes according to a curved amount of the curvedejection. In a case where the curved amount is small, even though thereare very-fine streaks, the streaks cannot be visually recognized byhuman eyes. On the other hand, in a case where the curved amount islarge, the streak can be visually recognized by human eyes. Since thereare almost no nozzles in which the curved amount of curved ejection isnonexistent, it is important to determine whether or not there is adefect based on a degree of visibility.

Further, a relationship between characteristics of a human visual systemand characteristics of an image capturing device is extremelynon-linear. For this reason, by the simple comparison processing basedon the difference described above, even though the correct answer datacomparison method is used, it is difficult to acquire defect informationthat is properly correlated with human visibility.

Therefore, by applying machine learning to the comparison processing(correct answer data comparison method) and using a human sensoryevaluation value as the learning output information, defect informationthat is properly correlated with human visibility can be acquired.

FIG. 7 is a block diagram illustrating a configuration of a machinelearning model generation system. The same portions as those in FIG. 1are denoted by the same reference numerals, and a detailed descriptionthereof will be omitted. FIG. 8 is a flowchart illustrating acomparison-processing machine learning model generation procedure.

A generation system 64 uses a comparison-processing machine learningmodel D12 in the comparison processing P1. The generation system 64includes a comparison-processing model generation device 66. Thecomparison-processing model generation device 66 includes acomparison-processing learning reference data storage unit 68, acomparison-processing learning inspection data storage unit 70, asensory evaluation value input unit 72, a comparison-processing modelgeneration unit 74, and a comparison-processing model storage unit 76.

The comparison-processing learning reference data storage unit 68 is amemory that stores comparison-processing learning reference data D9. Thecomparison-processing learning reference data D9 is data based on acaptured image obtained by capturing an image of a first printed matteras a reference by an image capturing device (not illustrated).

The comparison-processing learning inspection data storage unit 70 is amemory that stores comparison-processing learning inspection data D10.The comparison-processing learning inspection data D10 is data based ona captured image obtained by capturing an image of a second printedmatter as a comparison target by an image capturing device (notillustrated).

In the comparison-processing learning reference data D9 and thecomparison-processing learning inspection data D10, a pair of thecomparison-processing learning reference data D9 and thecomparison-processing learning inspection data D10 based on the sameprint digital data are present. That is, among printed matters printedbased on the same print digital data, the comparison-processing learningreference data D9, which is acquired from a captured image of a firstprinted matter without a defect as a reference, and thecomparison-processing learning inspection data D10, which is acquiredfrom a captured image of a second printed matter with a defect or asecond printed matter without a defect as a comparison target, form apair.

The sensory evaluation value input unit 72 is an input interface throughwhich a user inputs a sensory evaluation value D11 obtained by comparinga pair of the first printed matter and the second printed matter.

The comparison-processing model generation unit 74 is an arithmetic unitthat performs comparison-processing model generation processing P4. Thecomparison-processing model generation processing P4 is processing ofgenerating a comparison-processing machine learning model D12, from thecomparison-processing learning reference data D9 and thecomparison-processing learning inspection data D10 as learning inputinformation and the sensory evaluation value D11 as learning outputinformation. That is, the comparison-processing machine learning modelD12 is a model using an original data comparison method.

The comparison-processing model storage unit 76 is a memory that storesthe comparison-processing machine learning model D12 generated by thecomparison-processing model generation unit 74.

The generation system 64 configured as described above generates thecomparison-processing machine learning model D12 as follows. That is, auser inputs the sensory evaluation value D11 of the pair of the firstprinted matter and the second printed matter from the sensory evaluationvalue input unit 72 (an example of a sensory evaluation value inputstep). The comparison-processing model generation unit 74 performs thecomparison-processing model generation processing P4 of generating acomparison-processing machine learning model D12, from thecomparison-processing learning reference data D9 and thecomparison-processing learning inspection data D10 as learning inputinformation and the sensory evaluation value D11 as learning outputinformation (an example of a comparison-processing model generationstep).

The comparison-processing model generation processing P4 is performedbased on the correct answer data comparison method. Thus, it is easy toensure accuracy of the inspection, and it is possible to generate thecomparison-processing machine learning model D12 without using a machinelearning method that requires a large amount of data such as deeplearning.

As the comparison-processing model generation processing P4, forexample, multiple regression analysis, support vector machine, or apartial least-square method may be used. Compared to deep learning, thenumber of data required for one-digit or two-digit order is small, andthus it is easy to collect data by sensory evaluation. On the otherhand, even in a case where deep learning is used, a detection difficultylevel is lower than that in the original data comparison method, andthus an amount of data can be reduced.

The comparison processing unit 26 performs the comparison processing P1by using the comparison-processing machine learning model D12, and thusthe machine learning model D5 that is properly correlated with thesensory evaluation value and using the original data comparison methodcan be generated by the model generation unit 44.

Fourth Embodiment

[Effective Collection Method of Learning Defect Information]

The first learning reference data D1 and the learning inspection data D2are generated based on a captured image obtained by inputting thelearning print digital data, which is an acquisition source of thesecond learning reference data D4, to a printing device, and capturingan image of an output printed matter by an image capturing device. Onthe other hand, in this case, there is randomness in whether or not adefect occurs in the learning inspection data D2, and as a result, datacollection efficiency is extremely poor.

For example, in many cases, a streak-shaped defect that occurs in asingle-pass ink jet printing device occurs due to curved ejection fromnozzles. On the other hand, since it is difficult to intentionally causecurved ejection, it is necessary to perform printing many times untilcurved ejection occurs. For this reason, in a case of performingprinting for acquiring the learning inspection data D2, it is preferableto intentionally process a part of the image information of the learningprint digital data such that a pseudo defect is caused.

[Configuration of Printing Device and Collection of Learning DefectInformation]

FIG. 9 is a block diagram illustrating a configuration of a printingdevice including a learning information generation device. The sameportions as those in FIG. 1 are denoted by the same reference numerals,and a detailed description thereof will be omitted. FIG. 10 is aflowchart illustrating collection of the learning defect information.

A printing device 80 includes a raw print digital data storage unit 82,a print digital data processing unit 84, a processed print digital datastorage unit 86, a printing unit 88, and an image capturing unit 90, inaddition to the learning information generation device 20.

The raw print digital data storage unit 82 is a memory that stores rawprint digital data D13. Here, “raw” means that processing related to apseudo defect is not performed.

The print digital data processing unit 84 is an image processing unitthat intentionally processes at least a part of the image information ofthe raw print digital data D13 so as to cause a pseudo defect. Aposition, a type, and a degree of the pseudo defect to be caused may bedetermined by a user, or may be stored in a memory (not illustrated).The print digital data processing unit 84 generates processed printdigital data D14 by reading the raw print digital data D13 from the rawprint digital data storage unit 82 and performing desired processing onthe raw print digital data D13.

The processed print digital data storage unit 86 is a memory that storesthe processed print digital data D14. The printing device 80 may acquirethe processed print digital data D14 from an external device via aninterface (not illustrated).

The printing unit 88 is an image recording unit that prints an image ona recording medium based on the input print digital data. The printingunit 88 performs printing of the raw print digital data D13 and printingof the processed print digital data D14. The image capturing unit 90 isan image capturing device that captures an image of a printed matterprinted by the printing unit 88.

The printing unit 88 and the image capturing unit 90 perform printingand image-capturing processing P5. The printing and image-capturingprocessing P5 is processing of generating the first learning referencedata D1 by printing the raw print digital data D13 by the printing unit88 and capturing an image of a printed matter by the image capturingunit 90. The printing device 80 stores the first learning reference dataD1 in the first learning reference data storage unit 22.

Similarly, the printing unit 88 and the image capturing unit 90 performprinting and image-capturing processing P6. The printing andimage-capturing processing P6 is processing of generating the learninginspection data D2 with a defect by printing the processed print digitaldata D14 by the printing unit 88 and capturing an image of a printedmatter (an example of a printed matter with a defect) by the imagecapturing unit 90. The printing device 80 stores the learning inspectiondata D2 in the learning inspection data storage unit 24.

The comparison processing unit 26 performs the comparison processing P1by using the acquired first learning reference data D1 and the acquiredlearning inspection data D2, and thus it is possible to easily collectthe learning defect information D3 with a defect.

[Processing Method of Print Digital Data]

The printing unit 88 performs printing by ejecting inks of, for example,cyan (C), magenta (M), yellow (Y), and black (K). A case where the printdigital data is a CMYK 4-channel multi-valued digital image and a pixelvalue of each channel is a signal representing an ejection amount of acolor ink will be described as an example.

In a case of generating the print digital data D14 in which astreak-shaped defect due to an ejection failure of the black ink isexpressed at a certain position, the print digital data processing unit84 linearly decreases a pixel value of the K-channel digital image atthe position, and thus the streak-shaped defect can be expressed to bebrighter as compared with a case where there is no defect.Alternatively, by increasing the pixel value, the streak-shaped defectcan be expressed to be darker as compared with a case where there is nodefect. By changing a degree of increase or decrease of the pixel valueand a range in which a defect occurs, it is also possible to express adegree of a defect and a length of a defect. Of course, instead of theblack ink, in a case where a desired defect is expressed by processingchannel images of cyan, magenta, and yellow inks, there is no problem.

Alternatively, a streak-shaped defect may be expressed by changing acorrection amount for non-ejection correction or a degree of density ofnon-ejection nozzles. The ink jet printing has a function ofcompensating drawing at a non-ejection position by setting a defectivenozzle as a non-ejection nozzle and controlling an ejection amount of anadjacent nozzle (in many cases, increasing an ejection amount of anadjacent nozzle). On the other hand, in a case where a correction amountof the adjacent nozzle is not optimized, a streak-shaped defect mayoccur due to insufficient correction or overcorrection. Based on thisfact, by setting a certain nozzle as a non-ejection nozzle andperforming non-ejection correction which is not optimal, it is possibleto express a streak-shaped defect. Further, in a case where thenon-ejection nozzles are densely located and the adjacent nozzle cannotperform correction, a streak-shaped defect occurs. Based on this fact,by setting a certain nozzle and adjacent nozzles as non-ejection nozzlesand densely locating the non-ejection nozzles, it is possible to expressa streak-shaped defect. It can be said that changing of the correctionamount of the non-ejection correction and the degree of density of thenon-ejection nozzles means controlling of the image data (intermediateimage data) to be used in the printing machine after all. Thus, theimage data is included in an example of the processed print digital dataD14.

In a case of generating the print digital data D14 in which aspot-shaped defect due to ink dripping of the cyan ink is expressed at acertain position, the print digital data processing unit 84 increases apixel value of the C-channel digital image at the position in a circularform, and thus ink dripping of the cyan ink can be expressed. Bychanging a degree of increase or decrease of the pixel value and a rangein which a defect occurs, it is also possible to express a degree of adefect and a size of a defect. Of course, instead of the black ink, in acase where a desired defect is expressed by processing channel images ofcyan, magenta, and yellow inks, there is no problem.

In a case where the raw print digital data D13 is a binary digital imagehaving a value of 0 or 1 after halftone processing, instead ofincreasing or decreasing the pixel value, by increasing or decreasing anappearance frequency of a value of 0 or 1 at a desired position, it ispossible to express a defect.

By performing the printing and image-capturing processing P6 on theprocessed print digital data D14, it is possible to generate thelearning inspection data D2 with a defect. On the other hand, in orderto acquire the first learning reference data D1 paired with the learninginspection data D2, as in the example, the printing and image-capturingprocessing P5 may be performed on the raw print digital data D13.

Fifth Embodiment

[Printing Device for Generation of Machine Learning Model]

In the printing device, various paper brands are used as a recordingmedium. Further, depending on paper brands, characteristics of a printimage to be printed are changed. For example, in a case of an ink jetprinting device, permeability of an ink varies depending on paper, andas a result, reproduction density changes or a dot size of an ink onpaper changes. The changes affect a structure of the image.

In machine learning, as a model is used in a state closer to a conditionused in model learning, higher estimation accuracy can be obtained. Forexample, rather than in a case where a model generated using gloss paperA is applied to matte paper B, in a case where a model generated usingmatte paper B is applied to matte paper B or in a case where a modelgenerated using both of gloss paper A and matte paper B is applied tomatte paper B, higher estimation accuracy can be obtained.

On the other hand, it is difficult to generate a model in advance usingall paper to be used by customers of a printing machine. For thisreason, in an ink jet printing device and in a so-called digitalprinting machine which prints an electronic photograph or the like, forwhich it is preferable to generate a model according to a use conditionat a use place of a customer, a printing plate is not required andregistered print digital data can be immediately printed. Thus, byregistering, in the printing machine, in advance, various print digitaldata and print digital data in which a defect is expressed by processinga part of the print digital data, a model can be easily generated at ause place.

[Configuration of Printing Device and Generation of Machine LearningModel]

FIG. 11 is a block diagram illustrating a configuration of a printingdevice that generates a machine learning model. FIG. 12 is a flowchartillustrating generation of a machine learning model.

A printing device 92 includes a learning information generation device20, a model generation device 40, a printed-matter inspection device 50,a raw print digital data storage unit 82, a print digital dataprocessing unit 84, a processed print digital data storage unit 86, aprinting unit 88, and an image capturing unit 90.

The printing device 92 stores raw print digital data D13 in the rawprint digital data storage unit 82. The printing unit 88 and the imagecapturing unit 90 generate the first learning reference data D1 byperforming printing and image-capturing processing P5 on the raw printdigital data D13. The first learning reference data D1 is stored in thefirst learning reference data storage unit 22.

The print digital data processing unit 84 generates processed printdigital data D14 by reading the raw print digital data D13 from the rawprint digital data storage unit 82 and performing desired processing onthe raw print digital data D13. The processed print digital data D14 isstored in the processed print digital data storage unit 86.

The printing unit 88 and the image capturing unit 90 generate thelearning inspection data D2 by performing printing and image-capturingprocessing P6 on the processed print digital data D14. The learninginspection data D2 is stored in the learning inspection data storageunit 24.

The comparison processing unit 26 acquires learning defect informationD3D with a defect by performing comparison processing P1 on the firstlearning reference data D1 and the learning inspection data D2. Thelearning defect information D3D with a defect is stored in the learningdefect information storage unit 28.

The print digital data processing unit 84 generates print digital dataD14 in which various pseudo defects are caused. By repeating the sameprocessing, it is possible to collect a large number of the learningdefect information D3D with a defect.

The second learning reference data storage unit 30 stores the secondlearning reference data D4 which is the same as the raw print digitaldata D13. The second learning reference data storage unit 30 and the rawprint digital data storage unit 82 may be shared.

The information acquisition unit 42 acquires, as learning inputinformation, the learning inspection data D2 and the second learningreference data D4, and acquires, as learning output information, thelearning defect information D3D with a defect.

Further, the printing unit 88 and the image capturing unit 90 generatethe learning inspection data D2 by performing printing andimage-capturing processing P6 on the raw print digital data D13, and thelearning inspection data D2 is stored in the learning inspection datastorage unit. 24. The comparison processing unit 26 acquires thelearning defect information D3N without a defect by performing thecomparison processing P1 on the first learning reference data D1 and thelearning inspection data D2. In a case where a defect accidentallyoccurs during printing, it is noted that the learning defect informationD3D with a defect is acquired in the comparison processing P1.Alternatively, the information acquisition unit 42 acquires, as thelearning input information without a defect, the first learningreference data D1 based on the raw print digital data D13, and acquires,as the learning output information without a defect, the learning defectinformation D3N without a defect. As a method of acquiring the learningdefect information D3N, any one of the methods may be used, or both ofthe methods may be used in combination.

The model generation unit 44 performs adjusted model generationprocessing P7 of generating an adjusted machine learning model D15 usingthe original data comparison method, from the learning inspection dataD2 (in a case of the learning inspection data D2 without a defect, thefirst learning reference data D1 may be used) and the second learningreference data D4 as the learning input information and the learningdefect information D3D with a defect as the learning output information.The adjusted machine learning model D15 is stored in the model storageunit 46. Here, “adjustment” indicates that adjustment is performed for aspecific customer.

The adjusted model generation processing P7 may be performed using onlythe learning input information and the learning output informationacquired or generated in a use place (adjusted model generation methodA), or may be performed using a combination of the learning inputinformation and the learning output information, which are used in acase where the existing machine learning model D5 is created, and thelearning input information and the learning output information, whichare acquired or generated in a use place (adjusted model generationmethod B). Alternatively, the existing machine learning model may beupdated using a technique such as mini-batch learning or online learningbased on the learning input information and the learning outputinformation acquired or generated in a use place (adjusted modelgeneration method C). In this way, the adjusted machine learning modelD15 is generated.

In a case where the adjusted model generation method A is used, there isan advantage that a model specialized for a use condition of a customercan be generated. Further, in a case where the adjusted model generationmethod B or the adjusted model generation method C is used, there is anadvantage that performance of the generated machine learning model D5 iseasily stabilized because an amount of the learning data is large.

The feature of the fifth embodiment is to generate a machine learningmodel for the original data comparison method, which is adjusted for aspecific customer, by using the correct answer data comparison method(comparison processing P1). In the correct answer data comparisonmethod, defect inspection is performed based on the print image printedon paper of the same brand, and thus the inspection can be accuratelyperformed regardless of types of the used paper brands. On the otherhand, in the original data comparison method, comparison with the printdigital data is performed, and as a result, in a case where the machinelearning model is not adjusted for a customer, the inspection accuracytends to decrease. Therefore, as in the present embodiment, bygenerating a machine learning model for the original data comparisonmethod, which is adjusted for the paper brand used by a customer, byusing the correct answer data comparison method, the inspection accuracycan be greatly improved.

The fifth embodiment has been described focusing on the paper used by acustomer On the other hand, even in a case where the paper is the sameas the paper used for generation of the original machine learning model,“adjustment of the machine learning model” according to the presentembodiment is useful. For example, the adjustment of the machinelearning model is useful even in a case where a customer independentlychanges a configuration (an ink type, an ink ejection condition, adrying condition) related to printing. Further, even in a case where aconfiguration is not changed, “adjustment” can be performed according tovarious variations (individual differences) of the printing machineitself.

[Defect Inspection of Printed Matter]

In the printing device 92, the print digital data for a printed matterto be output is stored in the raw print digital data storage unit 82, asthe raw print digital data D13. The printing unit 88 prints the rawprint digital data D13. The image capturing unit 90 captures an image ofthe printed matter, and stores the captured image in the inspection datastorage unit 52, as inspection data D6.

Further, in the reference data storage unit 54, the print digital datafor the printed matter to be output, which is the same as the raw printdigital data D13, is stored as reference data D7. The reference datastorage unit 54 and the raw print digital data storage unit 82 may beshared.

The defect inspection unit 58 performs defect inspection processing P3of inspecting a defect of a printed matter by applying the inspectiondata D6 and the reference data D7, which are input, to the machinelearning model D5 and acquiring an inspection result D8. The acquiredinspection result D8 is stored in the inspection result storage unit 60.

Here, since the machine learning model D5 is updated by the adjustedmachine learning model D15, it is possible to perform the defectinspection according to the use condition of a customer. For example, bygenerating the adjusted machine learning model D15 based on the printedmatter obtained by performing printing on a paper brand used by acustomer and updating the machine learning model D5, it is possible toperform the defect inspection suitable for the paper brand.

In this way, in a state where the printing device 92 includes both ofthe learning information generation device 20 having an inspectionfunction using the correct answer data comparison method and theprinted-matter inspection device 50 having an inspection function usingthe original data comparison method, by providing a function ofgenerating or updating a machine learning model using the original datacomparison method based on an inspection result obtained by performinginspection using the correct answer data comparison method by a customerat a use place, the adjusted machine learning model D15 using theoriginal data comparison method, which is suitable for a use conditionof the customer, can be acquired at the use place.

In the present embodiment, arithmetic operations for the adjusted modelgeneration processing P7 are performed in the printing device 92. On theother hand, the arithmetic operations may be performed in a computerconnected to the printing device 92 or in a cloud environment connectedto a network.

Sixth Embodiment

An example in which an ink jet printing device is applied as theprinting device 92 will be described.

[Configuration of Ink Jet Printing Device]

FIG. 13 is an overall configuration diagram illustrating a schematicoverall configuration of an ink jet printing device 100. As illustratedin FIG. 13 , the ink jet printing device 100 is a printing machine thatprints a color image by ejecting four color inks of cyan (C), magenta(M), yellow (Y), and black (K) onto a sheet of paper P which is a printmedium.

As the paper P, general-purpose printing paper is used. Thegeneral-purpose printing paper is not so-called paper for ink jetexclusive use but paper including cellulose as a main component, such ascoated paper used for general offset printing. As the ink, an aqueousink is used. The aqueous ink is an ink in which a coloring material suchas a dye or a pigment is dissolved or dispersed in water or awater-soluble solvent.

As illustrated in FIG. 13 , the ink jet printing device 100 includes atransport unit 110, a printing unit 120, an image capturing unit 130, adrying unit 140, a sorting unit 150, and a paper discharge unit 160.

[Transport Unit]

The transport unit 110 transports paper P supplied from a paper supplyunit (not illustrated) in a transport direction (Y direction). Thetransport unit 110 includes an upstream pulley 112, a downstream pulley114, and a transport belt 116.

The upstream pulley 112 includes a rotation shaft (not illustrated)extending in a horizontal direction, and the rotation shaft is rotatablyand pivotally supported. The downstream pulley 114 includes a rotationshaft (not illustrated) parallel to the rotation shaft of the upstreampulley 112, and the rotation shaft is rotatably and pivotally supported.

The transport belt 116 is an endless belt made of stainless steel. Thetransport belt 116 is bridged between the upstream pulley 112 and thedownstream pulley 114. By using the transport belt 116 made of stainlesssteel, flatness of the paper P can be kept good.

The downstream pulley 114 includes a motor (not illustrated) as adriving unit. In a case where the motor is driven, the downstream pulley114 rotates counterclockwise in FIG. 13 . The upstream pulley 112 isdriven by the rotation of the downstream pulley 114, and rotatescounterclockwise in FIG. 13 . By the rotation of the upstream pulley 112and the downstream pulley 114, the transport belt 116 travels betweenthe upstream pulley 112 and the downstream pulley 114 along a travelroute.

The paper P supplied from a paper supply unit (not illustrated) isplaced on a transport surface of the transport belt 116. The transportunit 110 transports the paper P placed on the transport belt 116 along atransport path from the upstream pulley 112 to the downstream pulley114, and delivers the paper P to the paper discharge unit 160. At aposition on the transport path that faces the printing unit 120, theimage capturing unit 130, the drying unit 140, and the sorting unit 150,the paper P is transported in a state where the print surface ishorizontally maintained.

By providing a plurality of suction holes (not illustrated) on thetransport belt 116 and sucking the suction holes of the transport belt116 by a pump (not illustrated), the paper P placed on the transportsurface of the transport belt 116 may be sucked and maintained on thetransport surface.

[Printing Unit]

The printing unit 120 forms (prints) an image on the paper P. Theprinting unit 120 includes ink jet heads 122C, 122M, 122Y, and 122K. Theink jet head 122C ejects cyan ink droplets by an ink jet method.Similarly, the ink jet heads 122M, 122Y, and 122K respectively ejectmagenta, yellow, and black ink droplets by the ink jet method.

The ink jet heads 122C, 122M, 122Y, and 122K are disposed at regularintervals along the transport path of the paper P by the transport belt116. Each of the ink jet heads 122C, 122M, 122Y, and 122K is configuredwith a line head, and has a length corresponding to a maximum paperwidth. The ink jet heads 122C, 122M, 122Y, and 122K are disposed suchthat a nozzle surface (a surface on which the nozzles are arranged)faces the transport belt 116.

The ink jet heads 122C, 122M, 122Y, and 122K form an image on the printsurface of the paper P by ejecting ink droplets from the nozzles formedon the nozzle surface toward the paper P transported by the transportbelt 116.

In this way, the printing unit 120 generates a printed matter byscanning the paper P transported by the transport belt 116 once, thatis, by a so-called single-pass method. The printing unit 120 may becommon to the printing unit 88.

[Image Capturing Unit]

The image capturing unit 130 may be common to the image capturing unit90. The image capturing unit 130 acquires an image on the print surfaceof the paper P. The image capturing unit 130 is disposed on thedownstream side of the printing unit 120 with respect to the transportdirection of the paper P. The image capturing unit 130 includes ascanner 132.

The scanner 132 is a device that optically reads an image, which isformed on the paper P by using the ink jet heads 122C, 122M, 122Y, and122K, and generates image data indicating the read image. The scanner132 includes an imaging device that images an image printed on the paperP and converts the image into an electric signal. As the imaging device,a color charge coupled device (CCD) linear image sensor may be used.Instead of the color CCD linear image sensor, a color complementarymetal oxide semiconductor (CMOS) linear image sensor may be used.

The scanner 132 may include, in addition to the imaging device, anillumination optical system that illuminates a reading target and asignal processing circuit that generates digital image data byprocessing a signal obtained from the imaging device.

[Drying Unit]

The drying unit 140 dries the ink on the paper P. The drying unit 140 isdisposed on the downstream side of the image capturing unit 130 withrespect to the transport direction of the paper P.

The drying unit 140 includes a heater 142. As the heater 142, forexample, at least one of a halogen heater or an infrared heater is used.The heater 142 dries the ink on the paper P by heating the print surfaceof the paper P. The drying unit 140 may include a blowing unit such as afan or a blower.

[Sorting Unit]

The sorting unit 150 sorts a printed matter according to qualitydetermination on the paper P transported by the transport belt 116. Thesorting unit 150 is disposed on the downstream side of the drying unit140 with respect to the transport direction of the paper P. The sortingunit 150 includes a stamper 152.

The stamper 152 performs stamping processing of applying an ink onto aleading edge of the paper P that is determined as a defective printedmatter according to the quality determination on the paper P transportedby the transport belt 116.

[Paper Discharge Unit]

The paper discharge unit 160 collects the dried paper P (printed matter)on which an image is formed. The paper discharge unit 160 is disposed onthe downstream side of the sorting unit 150 with respect to thetransport direction of the paper P and at the end point of the transportpath of the transport unit 110. The paper discharge unit 160 includes apaper discharge tray 162.

The paper discharge tray 162 stacks and collects the paper P transportedby the transport belt 116. The paper discharge tray 162 includes a frontpaper pad, a rear paper pad, and a horizontal paper pad (notillustrated), and thus the paper P is stacked in an orderly manner.

Further, the paper discharge tray 162 is provided so as to be able tomove up and down by a lifting device (not illustrated). The driving ofthe lifting device is controlled in association with an increase or adecrease of the paper P stacked on the paper discharge tray 162.Thereby, the paper P located at the highest position among the paper Pstacked on the paper discharge tray 162 always has a constant height.

[Control System of Ink Jet Printing Device]

FIG. 14 is a block diagram illustrating an internal configuration of anink jet printing device 100. The ink jet printing device 100 includes,in addition to the learning information generation device 20, the modelgeneration device 40, the printed-matter inspection device 50, thetransport unit 110, the printing unit 120, the image capturing unit 130,the drying unit 140, the sorting unit 150, and the paper discharge unit160, a user interface 170, a storage unit 172, an integrative controlunit 174, a transport control unit 176, a print control unit 178, animage capturing control unit 180, a drying control unit 182, a sortingcontrol unit 184, and a paper discharge control unit 186.

The user interface 170 includes an input unit (not illustrated) and adisplay unit (not illustrated) that allow the user to operate the inkjet printing device 100. The input unit is, for example, an operationpanel that receives an input from a user. The display unit is, forexample, a display that displays image data and various information. Theuser can cause the ink jet printing device 100 to print a desired imageby operating the user interface 170.

The storage unit 172 stores a program for controlling the ink jetprinting device 100 and information required for executing the program.The storage unit 172 is configured with a hard disk (not illustrated) ora non-transitory recording medium such as various semiconductormemories. A volatile memory such as a random access memory (RAM) (notillustrated) that temporarily stores the first learning reference dataD1, the learning inspection data D2, the learning defect information D3,the second learning reference data D4, the machine learning model D5,and the like may be used. The storage unit 172 also serves as the rawprint digital data storage unit 82 and the processed print digital datastorage unit 86.

The integrative control unit 174 performs various processing accordingto the program stored in the storage unit 172, and performs integrativecontrol of the overall operation of the ink jet printing device 100. Theintegrative control unit 174 also performs integrative control of thelearning information generation device 20, the model generation device40, and the printed-matter inspection device 50. The integrative controlunit 174 also serves as the print digital data processing unit 84.

The transport control unit 176 causes the transport unit 110 totransport the paper P in the transport direction by controlling a motor(not illustrated) of the transport unit 110. Thereby, the paper Psupplied from a paper supply unit (not illustrated) passes throughpositions facing the printing unit 120, the image capturing unit 130,the drying unit 140, and the sorting unit 150, and is finally dischargedto the paper discharge unit 160.

The print control unit 178 controls ejection of inks by the ink jetheads 122C, 122M, 122Y, and 122K. The print control unit 178 causes theink jet heads 122C, 122M, 122Y, and 122K to eject cyan, magenta, yellow,and black ink droplets onto the paper P at timings at which the paper Ppasses through positions facing each nozzle surface. Thereby, a colorimage is formed on the print surface of the paper P, and thus the paperP becomes a “printed matter”.

The image capturing control unit 180 causes the image capturing unit 130to read the image of the paper P (printed matter) by controlling imagingby the scanner 132. The image capturing control unit 180 causes thescanner 132 to read the image formed on the paper P at a timing at whichthe paper P passes through a position facing the scanner 132. Thereby,the inspection image is acquired.

The drying control unit 182 causes the drying unit 140 to dry the paperP by controlling heating by the heater 142. The drying control unit 182causes the heater 142 to heat the paper P in a case where the paper Ppasses through a position facing the heater 142.

The sorting control unit 184 causes the sorting unit 150 to sort thepaper P by controlling stamping processing by the stamper 152. Thesorting control unit 184 (an example of an output unit that outputs adetection result of a defect of a printed matter) classifies the printedmatter into a non-defective printed matter and a defective printedmatter according to the detected defect. In a case where the paper Ppassing through a position facing the stamper 152 is the paper Pdetermined as a defective printed matter, the sorting control unit 184performs stamp processing by the stamper 152.

The paper discharge control unit 186 controls stacking of the paper P bythe paper discharge tray 162. The paper P is discharged onto the paperdischarge tray 162, and is stacked. An ink is applied on the leadingedge of the paper P as a defective printed matter. Therefore, a user canspecify the defective printed matter among the paper P stacked on thepaper discharge tray 162.

FIG. 15 is a diagram illustrating an example of a printed matter whichis printed by an ink jet printing device 100. As illustrated in FIG. 15, a defective nozzle detection pattern PT and an image G are printed onthe paper P as a printed matter.

The defective nozzle detection pattern PT includes lines that are spacedat regular intervals in an X direction and are extended along a Ydirection. By printing a plurality of defective nozzle detectionpatterns PT by shifting the nozzles forming the lines one by one, it ispossible to detect defects of all the nozzles.

The image G is a print result which is printed based on the printdigital data. The printed-matter inspection device 50 detects a defectof the image G. The image G illustrated in FIG. 15 has a streak-shapeddefect DF extending in the Y direction. Examples of the streak-shapeddefect include not only continuous streak-shaped defects but alsointermittent streak-shaped defects. The defect DF occurs because thenozzle that ejects an ink onto a position of the defect DF is defectivein ejection. The defective ejection nozzle may be detected by using thedefective nozzle detection pattern PT. In the ink jet printing device100, the printed matter is classified into a non-defective printedmatter and a defective printed matter according to the defect detectedby the printed-matter inspection device 50.

Others

The machine learning model generation method and the inspection methodmay be realized as a program for causing a computer to execute eachstep, and a non-transitory recording medium such as a compact disk-readonly memory (CD-ROM) that stores the program may be configured.

In the embodiments described above, for example, as a hardware structureof a processing unit that executes various processing such as processingin the learning information generation device 20, the model generationdevice 40, and the printed-matter inspection device 50, the followingvarious processors may be used. The various processors include, asdescribed above, a CPU, which is a general-purpose processor thatfunctions as various processing units by executing software (program),and a dedicated electric circuit, which is a processor having a circuitconfiguration specifically designed to execute a specific process, suchas a programmable logic device (PLD) or an application specificintegrated circuit (ASIC) that is a processor of which the circuitconfiguration may be changed after manufacturing such as a graphicsprocessing unit (GPU) or a field programmable gate array (FPGA), whichis a processor specialized for image processing.

One processing unit may be configured by one of these variousprocessors, or may be configured by a combination of two or moreprocessors of the same type or different types (for example, acombination of a plurality of FPGAs, a combination of a CPU and an FPGA,or a combination of a CPU and a GPU). Further, the plurality ofprocessing units may be configured by one processor. As an example inwhich the plurality of processing units are configured by one processor,firstly, as represented by a computer such as a client and a server, aform in which one processor is configured by a combination of one ormore CPUs and software and the processor functions as the plurality ofprocessing units may be used. Secondly, as represented by a system onchip (SoC) or the like, a form in which a processor that realizes thefunction of the entire system including the plurality of processingunits by one integrated circuit (IC) chip is used may be used. Asdescribed above, the various processing units are configured by usingone or more various processors as a hardware structure.

Further, as the hardware structure of the various processors, morespecifically, an electric circuit (circuitry) in which circuit elementssuch as semiconductor elements are combined may be used.

The technical scope of the present disclosure is not limited to thescope described in the above embodiments. The configurations and thelike in the embodiments may be appropriately combined with each otherwithout departing from the spirit of the present disclosure.

EXPLANATION OF REFERENCES

-   -   10: generation system    -   20: learning information generation device    -   22: first learning reference data storage unit    -   24: learning inspection data storage unit    -   26: comparison processing unit    -   28: learning defect information storage unit    -   30: second learning reference data storage unit    -   40: model generation device    -   42: information acquisition unit    -   44: model generation unit    -   46: model storage unit    -   50: printed-matter inspection device    -   52: inspection data storage unit    -   54: reference data storage unit    -   56: model storage unit    -   58: defect inspection unit    -   60: inspection result storage unit    -   62: generation system    -   64: generation system    -   66: comparison-processing model generation device    -   68: comparison-processing learning reference data storage unit    -   70: comparison-processing learning inspection data storage unit    -   72: sensory evaluation value input unit    -   74: comparison-processing model generation unit    -   76: comparison-processing model storage unit    -   80: printing device    -   82: raw print digital data storage unit    -   84: print digital data processing unit    -   86: processed print digital data storage unit    -   88: printing unit    -   90: image capturing unit    -   92: printing device    -   100: ink jet printing device    -   110: transport unit    -   112: upstream pulley    -   114: downstream pulley    -   116: transport belt    -   120: printing unit    -   122C: ink jet head    -   122K: ink jet head    -   122M: ink jet head    -   122Y: ink jet head    -   130: image capturing unit    -   132: scanner    -   140: drying unit    -   142: heater    -   150: sorting unit    -   152: stamper    -   160: paper discharge unit    -   162: paper discharge tray    -   170: user interface    -   172: storage unit    -   174: integrative control unit    -   176: transport control unit    -   178: print control unit    -   180: image capturing control unit    -   182: drying control unit    -   184: sorting control unit    -   186: paper discharge control unit    -   D1: first learning reference data    -   D2: learning inspection data    -   D3: learning defect information    -   D3D: learning defect information with defect    -   D3N: learning defect information without defect    -   D4: second learning reference data    -   D5: machine learning model    -   D6: inspection data    -   D7: reference data    -   D8: inspection result    -   D9: comparison-processing learning reference data    -   D10: comparison-processing learning inspection data    -   D11: sensory evaluation value    -   D12: comparison-processing machine learning model    -   D13: raw print digital data    -   D14: processed print digital data    -   D15: adjusted machine learning model    -   P: paper    -   P1: comparison processing    -   P2: model generation processing    -   P3: defect inspection processing    -   P4: comparison-processing model generation processing    -   P5: printing and image-capturing processing    -   P6: printing and image-capturing processing    -   P7: adjusted model generation processing

What is claimed is:
 1. A machine learning model generation method fordetecting a defect of a printed matter by comparing, using a machinelearning model, inspection data which is acquired based on a capturedimage obtained by capturing an image of the printed matter and referencedata which is acquired based on print digital data, the methodcomprising: an acquisition step of acquiring learning inspection datathat is based on a captured image obtained by capturing an image of aprinted matter as an inspection target which is printed based onlearning print digital data, learning defect information of the learninginspection data that is estimated by performing comparison processing offirst learning reference data and the learning inspection data, thefirst learning reference data being based on a captured image obtainedby capturing an image of a printed matter as a reference which isprinted based on the learning print digital data, and second learningreference data based on the learning print digital data; and ageneration step of generating the machine learning model by using atleast the learning inspection data and the second learning referencedata as learning input information and using at least the learningdefect information as learning output information.
 2. The machinelearning model generation method according to claim 1, wherein thelearning defect information includes a discrete value, and in thegeneration step, the machine learning model for performingclassification is generated.
 3. The machine learning model generationmethod according to claim 2, wherein the discrete value is a binarydiscrete value indicating the presence or absence of the defect of theprinted matter.
 4. The machine learning model generation methodaccording to claim 2, wherein the discrete value is a ternary or higherdiscrete value indicating a degree of the defect of the printed matter.5. The machine learning model generation method according to claim 1,wherein the learning defect information includes a continuous value, andin the generation step, the machine learning model for performingregression is generated.
 6. The machine learning model generation methodaccording to claim 1, wherein the learning defect information includesposition information of the defect of the printed matter.
 7. The machinelearning model generation method according to claim 1, wherein, in thegeneration step, the machine learning model is generated by deeplearning.
 8. The machine learning model generation method according toclaim 1, wherein, in the generation step, the machine learning model isgenerated by using at least the first learning reference data and thesecond learning reference data as learning input information and usingat least the learning defect information indicating that a defect doesnot exist as learning output information.
 9. The machine learning modelgeneration method according to claim 1, wherein, in the acquisitionstep, at least the learning inspection data is acquired, the learninginspection data being obtained by capturing an image of a printed matterprinted based on processed print digital data in which a defect isexpressed by processing at least a part of the learning print digitaldata.
 10. The machine learning model generation method according toclaim 1, wherein, in the acquisition step, the first learning referencedata is acquired, and the method further comprises a comparisonprocessing step of estimating the learning defect information of thelearning inspection data by performing comparison processing of thelearning inspection data and the first learning reference data.
 11. Themachine learning model generation method according to claim 10, wherein,in the comparison processing step, comparison processing is performed byusing a comparison-processing machine learning model.
 12. The machinelearning model generation method according to claim 11, furthercomprising: a sensory evaluation value input step of inputting a sensoryevaluation value obtained by comparing a first printed matter as areference which is printed based on the learning print digital data anda second printed matter as a comparison target which is printed based onthe learning print digital data; and a comparison-processing modelgeneration step of generating the comparison-processing machine learningmodel by using comparison-processing learning reference data obtained bycapturing an image of the first printed matter and comparison-processinglearning inspection data obtained by capturing an image of the secondprinted matter as learning input information and using the sensoryevaluation value as learning output information.
 13. An inspectionmethod comprising: a defect inspection step of acquiring inspection databased on a captured image obtained by capturing an image of a printedmatter as an inspection target which is printed based on print digitaldata and reference data based on the print digital data and detecting adefect of the printed matter as the inspection target by comparing theinspection data and the reference data by using the machine learningmodel generated by the machine learning model generation methodaccording to claim
 1. 14. A machine learning model generation device fordetecting a defect of a printed matter by comparing, using a machinelearning model, inspection data which is acquired based on a capturedimage obtained by capturing an image of the printed matter and referencedata which is acquired based on print digital data, the devicecomprising: an acquisition unit that acquires learning inspection datathat is based on a captured image obtained by capturing an image of aprinted matter as an inspection target which is printed based onlearning print digital data, learning defect information of the learninginspection data that is estimated by performing comparison processing offirst learning reference data and the learning inspection data, thefirst learning reference data being based on a captured image obtainedby capturing an image of a printed matter as a reference which isprinted based on the learning print digital data, and second learningreference data based on the learning print digital data; and ageneration unit that generates the machine learning model by using atleast the learning inspection data and the second learning referencedata as learning input information and using at least the learningdefect information as learning output information.
 15. An inspectiondevice comprising: the machine learning model generation deviceaccording to claim 14; and a defect inspection unit that acquiresinspection data based on a captured image obtained by capturing an imageof a printed matter as an inspection target which is printed based onprint digital data and reference data based on the print digital dataand detects a defect of the printed matter as the inspection target bycomparing the inspection data and the reference data by using a machinelearning model generated by the machine learning model generationdevice.
 16. A printing device comprising: the inspection deviceaccording to claim 15; a printing unit that generates a printed matterby performing printing based on the print digital data; a camera thatcaptures an image of the printed matter; and an output unit that outputsa detection result of a defect of the printed matter.
 17. The printingdevice according to claim 16, further comprising: a processing unit thatgenerates processed print digital data in which a defect is expressed byprocessing at least a part of the learning print digital data, whereinthe printing unit generates a printed matter with a defect by performingprinting based on the processed print digital data, the camera capturesan image of the printed matter with a defect, and the acquisition unitacquires, as the learning inspection data, at least data based on acaptured image obtained by capturing an image of the printed matter witha defect.
 18. The printing device according to claim 16, wherein thegeneration unit generates an adjusted machine learning model suitablefor the printing device by adjusting a machine learning model by usinglearning inspection data based on a captured image obtained by capturingan image of a printed matter by the camera.
 19. The printing deviceaccording to claim 16, wherein the printing unit performs printing usingan ink jet head.
 20. A non-transitory and computer-readable recordingmedium that causes a computer to execute the machine learning modelgeneration method according to claim 1 in a case where a command storedin the recording medium is read by the computer.